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ABSTRACT 

The IPSO framework allows optimal design of experiments and surveys. We discuss the utility of 
IPSO with a simplified 10 parameter MCMC D-optimisation of a dark energy survey. The resulting 
optimal number of redshift bins is typically two or three, all situated at z < 2. By exploiting optimi- 
sation we show how the statistical power of the survey is significantly enhanced. Experiment design is 
aided by the richness of the figure of merit landscape which shows strong degeneracies, which means 
one can impose secondary optimisation criteria at little cost. For example, one may choose either to 
maximally test a single model (e.g. ACDM) or to get the best model-independent constraints possible 
(e.g. on a whole space of dark energy models). Such bifurcations point to a future where cosmological 
experiments become increasingly specialised and optimisation increasingly important. 
Subject headings: cosmological parameters — large-scale structure of universe — surveys 



1. INTRODUCTION 

We have reached an enviable resonance in which im- 
provements in detector performance and cost are allow- 
ing not only rapid gains in our fundamental knowledge 
of the cosmos but also the opportunity for smaller ex- 
periments to make critical contributions to that knowl- 
edge. This has resulted in a surge of interest in next- 
generation experiment design with over twenty major 
surveys in planning or construction in observational cos- 
mology alone. Experimental cosmology has changed in a 
few short years into a crowded and jostling marketplace. 

There are several big prizes currently at stake: De- 
tection of dark energy dynamics, B-mode polarisation 
and cosmological non-Gaussianity. Competition, lim- 
ited funding, low signal-to-noise and extreme competi- 
tion mean that new surveys will need to be increasingly 
optimised to get the most out of them. The aims of this 
Letter are to show how this can be achieved in a cross- 
disciplinary way and to illustrate some of the rich aspects 
of cosmological optimisation. 

2. IPSO 

Integrated Parameter Space Optimisation (IPSO; Bas- 
sett 2004, hereafter B04) proceeds by first constructing a 
class of candidate survey /experiment geometries, la- 
beled by survey parameters, 5^, such as areal and redshift 
coverage. 

Second, a target parameter space, 0, is defined, con- 
sisting of the parameters that we wish to optimally con- 
strain (labeled There are also typically nuisance 
parameters we need to marginalise over (labeled (pa,b...)' 

Third, a Figure of Merit (FoM) is defined which as- 
signs a single real number to each candidate survey. The 
candidate with the extremal FoM is the optimal experi- 
ment/survey. The FoM we consider is defined by (B04): 



FoM{s^) = [ I{s,,0)p{0)dO. 



(1) 



function' that weights the different regions of the param- 
eter space. By integrating over the parameter space we 
do not make assumptions about the underlying model, 
which is particularly important when we have very lim- 
ited knowledge of the underlying physics, as is the case 
with dark energy. 

Most choices for /(s^, 0) typically invoke either the pa- 
rameter covariance matrix or F, the Fisher matrix, de- 
fined by: 

/ dHnC \ ^ f dX dX^ 



Fab 



KdOAdOB 



E 



'is)- (2) 



I{si^ 0) is a scalar which depends on the survey geometry 
(through the s^), and position in andp(6>) is a 'window 



Here we use A = {/i...,a...} to label both fundamen- 
tal and nuisance parameters, C is the likelihood, X = 
Ci^dL^H represents the quantity being measured with 
i labeling redshift bin or Fourier mode as appropriate 
(Tegmark et al. 1998). The ef are the error variances on 
X and depend explicitly on the survey parameters, s^, 
unlike the derivatives, dX/dOA- In computing integrals 
such as this allows for significant cpu gains since the 
derivatives need only to be computed once. 

Via the Cramer-Rao bound provides the best pos- 
sible covariance matrix and hence a lower bound on 
the achievable parameter variances. Although there are 
many choices for /(s^, 0^) (B04) we focus on only one for 
simplicity: D-optimality, defined by 

7(5, 0^) = log det(F + P) D - optimality (3) 

where 'det' denotes matrix determinant and P is the 
prior precision matrix, viz. the Fisher matrix of all the 
relevant prior data. 

Eq. is the gain in Shannon information or entropy 
over the prior. Maximising provides the best possible 
gain in constraints on the parameters 0^ over what was 
available from just the prior data, P. It is known as D- 
optimality in the design literature. If P = maximising 
is equivalent to minimising the volume of the error 
ellipses, an alternative FoM (Huterer and Turner 2001, 
Frieman et al. 2003, B04). Via the General Equivalence 
Theorem, D-Optimal solutions are also optimal under 
other FoM. For these reasons it seems appropriate for 
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Fig. 1. — Typical D-optimality improvements on er- 
ror ellipses at the two base points (wq^wi) = ( — 1.3, —0.2) and 
(—0.7, 0.2). The unoptimised survey (outer dark eUipses) has errors 
between 50% and three times larger in both wq and wi than the 
D-optimal survey (inner hght eUipse) which was optimised around 
ACDM. Note the particularly significant gains in the phantom re- 
gion Wq < — 1. 



cosmological applications, although as we will see, sec- 
ondary optimisation criteria can be imposed at almost 
no cost to the primary FoM. 

Nuisance parameters, such as ftm etc., whose val- 
ues we do not know precisely but which we do not want 
to optimise with respect to, can be easily dealt with by 
inverting the full Fisher matrix Fab , extracting the rele- 
vant submatrix corresponding to the re-inverting (e.g. 
Seo & Eisenstein 2003, B04) and then applying Eq. Q. 
Further, any reasonable FoM can also be generalised to 
allow inclusion of competing surveys by simply replacing 
F ^ F + ^ where is the sum of the Fisher matrices ex- 
pected for the competing surveys. In this way IPSO will 
find the optimal niche with respect to the other surveys 
(B04). 

3. OPTIMIZING CMB AND WEAK LENSING SURVEYS 

When is optimisation worth doing? To illustrate this 
let us contrast weak lensing (wl) convergence and CMB 
surveys on the celestial sphere. In both of these cases the 
Fisher matrix is a sum over i (Hu and Tegmark 1999, 
Knox and Song 2002, Kesden et al 2002): 
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where X = CMB, wl, Nf is the total noise for the sur- 
vey and fsky is the fraction of the sky observed. For the 
CMB we consider only one spectrum (e.g. the B-mode 
power spectrum). In both cases we assume that the sur- 
veys are constrained to last a given length of time, T, 
and ask 'what is the optimal sky coverage, fsky^ given 
this constraint?' For CMB experiments we have (Knox 
1995): 



1/2 MB 



^fsky 



(5) 



where T = tpixNpix is the length of the survey, a 
is a proportionality constant and the Npix pixels are 
each observed for time tpix using a Gaussian beam with 
FWHM (X <J5. The first (second) term in Q is the noise 
from sample variance (instrument noise). 

CMB experiments will benefit from optimisation since 
the competition between the terms in eq. (|5J creates a 



local minimum in the noise (Jaffe et al 1999). To apply 
IPSO to the CMB one must first choose 0. For example, 
for optimal detection of deviations from the infiationary 
consistency conditions the key variable is ^ = + r/4.8 
where rit is the tensor spectral index and r is the ratio 
of tensor to scalar quadrupole in the CMB. Single field 
infiation predicts this should vanish. Hence a high-a de- 
tection of would put severe pressure on simple in- 
fiationary models. In contrast, an experiment designed 
to detect B-mode polarisation alone would optimise to 
detect r only and would lead to a different optimal area. 
In contrast, for weak lensing (Kaiser 1992) 
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where ~ 0.35 is the approximately constant intrin- 
sic ellipticity error and the surface density of detected 
galaxies scales roughly as nV^ where t is the integration 
time per field of view. The noise terms differ cru- 
cially when it comes to optimisation of the areal coverage, 
fsky Unlike the CMB noise, N^^ has no local minimum; 
the weak lensing Fisher matrix is a monotonic function 
of fsky Optimising any of the FoM simply proceeds by 
using the largest feasible area to minimise the sample 
variance. 

If, in addition, the intrinsic ellipticity noise dominates 
the noise (as it does for the proposed SNAP WAS) then 
the FoM becomes essentially independent of fsky and the 
gain of going to the largest area is minimal, as found by 
(Rhodes et al, 2004). 

4. OPTIMAL MEASUREMENTS OF THE HUBBLE 
CONSTANT 

To illustrate some of the issues one faces in applying 
IPSO to realistic surveys, consider the optimisation of 
a redshift survey designed to measure the Hubble con- 
stant through observation of the radial baryon oscilla- 
tions (Seo & Eisenstein 2003, Blake & Glazebrook 2003, 
Linder 2003, Amendola et al. 2004, Yamamoto et al. 
2005). For clarity we assume no nuisance parameters, a 
fiat FLRW model with Qrn = 0.3, Hq known exactly and 
we ignore the constraints from dA which a full optimisa- 
tion would include. 

We consider a model of dark energy based on Taylor 
expansion in powers of (1 — a) = (Chevallier M., 
Polarski 2001, Linder 2003, Bassett, Corasaniti & Kunz 
2004), with w = Pde/pde 

^2 
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l^z (1 + ^)^ 
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2(l + z)2 

where Pde{z) is the dark energy energy density. 

Comparing with eq. X = E = H/Hq. Rather than 
the optimal area, fsky^ we want the optimal number of 
redshift bins, N, what redshifts they should be centered 
on, Zi, and how long we should observe in each bin, ti. 
Again we assume fixed total survey time (T) so we need 

to optimise given the constraint ti = T. 

We assume that the error bars scale as e~ = e^t^ 
where ex {1 -\- Zi)~^ gives the efficiency with which 
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Fig. 2. — Error on wq (ctwo) versus area of the error 

ellipse. Note the wide range of at almost constant minimum 
area, cr^^^ effectively measures the ellipticity of the error ellipse. At 
nearly constant FoM one can optimise to obtain circular or very 
thin ellipses, depending on one's aims. The size of the points is 
proportional to the error in wi. 



galaxies are detected and 7, /3 parameterise our igno- 
rance. These could be treated as nuisance parameters to 
be marginalised over but we find that our main results 
are insensitive to both over the range /? = 1 — 2,7 = 1 — 2 
we consider. Note that 7 = 1 implies that the FoM is 
maximised on the boundary of the allowed redshift re- 
gion (just as was the case with the weak lensing survey 
earlier). We focus on the case 7 = 2 here for illustrative 
purposes. The real constraint will be significantly more 
complex and we leave this issue to future work. 

We restrict the bin redshifts to be between 0.5 < Zi < 
4.5 which is a feasible range for future baryon oscilla- 
tion surveys such as KAOS and set W2 = for clarity. 
We performed an MCMC (Christensen and Meyer, 2000) 
optimisation with the D-Optimal FoM Q with ten free 
survey parameters: {U^Zi} giving the integration time 
and redshifts of the five bins. The effective number of 
bins varies dynamically because the MCMC chain could 
(and typically did) assign negligible amounts of observ- 
ing time to some of the bins. 

We ran multiple chains (up to 5000) with random start- 
ing configurations for the survey and used a standard 
Hastings-Metropolis algorithm for jump acceptance. In- 
stead of directly performing the integral Q we used the 

definition FoM{s) = ;^ X]i=i ^(^^^ ^a) where the ^a^re 
drawn randomly from a probability distribution based 
on p{Oi) in We chose p to be a bi-variate Gaussian 
centered on the ACDM point k;o = — 1, '^1 = so our 
optimisation was chosen to detect slowly varying dark 
energy dynamics close to a cosmological constant. 

The Fisher matrix derivatives based on eq. Q are 
simple to compute; e.g. 
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dwi '"'^ ' 1 + z 

Our unoptimised fiducial survey had five redshift bins 
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Fig. 3. — Choices in optimisation. At constant FoM one can 
optimise to achieve thin error eUipses with tiny transverse errors 
(diagonal dark blue "strip") or to achieve the best joint constraints 
on all parameters (light blue ellipse). All ellipses are computed at 
W{) = —l^wi = and reduce the total unoptimised ellipse area 
(dark blue ellipse) by about 600%. 




Fig. 4. — Redshifts and integration times for the optimal 
surveys shown in Fig. (|^ and the resulting error bars on 
H(z). Top Left - D-optimal maximum FoM (which also has 
the minimum error ellipse area). It splits the total survey equally 
between z = 0.5 and z = 1.15. Top Right - the thinnest possible 
ellipse with all measurements at a single redshift, z = 0.5. The 
bottom two panels show the geometries for the 2nd (left) and 
2000th (right) largest FoM (only 0.005% and 6% smaller than the 
maximum respectively). This shows the diversity of geometries 
with nearly degenerate FoM. 



located at Zi = 0.6,0.8, 1, 1.2,3, as in (Seo & Eisenstein 
2003 and Amendola et al. 2004), with equal integration 
time (T/5) assigned to each bin. 

Fig. Q shows typical gains over the unoptimised sur- 
vey for a near optimal survey chosen randomly from the 
5000 MCMC chains while Fig. shows the area of the 
error ellipse versus the corresponding error on wq for each 
of the 5000 locally optimal solutions. It is very clear that 
at almost identical area (and FoM) there is a very wide 
range of error ellipse ellipticity (controlled by (Jwq)- In 
other words, there are many local maxima which come 
very close to matching the global maximum. 

The implications of this FoM 'degeneracy' for ruling 
out dark energy models are clarified in Fig. ^ where 
we show the thinnest error ellipse (diagonal "strip"), the 
unoptimised error ellipse and the error ellipse with the 
maximal FoM, all computed at the ACDM point (the 
thinnest ellipse is shifted down for clarity). 

This degeneracy offers the chance for secondary opti- 
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Fig. 5. — Phase plot of the redshifts, 2:1, 2:2, of the two dominant 
bins with the most integration time for 5000 separate MCMC opti- 
misation chains. The dominant bins are typically found at 2:1,2 < 2 
although surveys with 2:2 > 2 can be competitive. Note the clus- 
tering around the minimum redshift we considered, z = 0.5. The 
diagonal band results in very thin error ellipse (large ctwq in figure 
0. 



misation (the primary one in this case being based on 
the D-optimal FoM). For example, one could choose ge- 
ometries that deliver the best constraints on a particular 
linear combination of the parameters 0^ adapted to the 
degeneracy structure of the observations while sacrificing 
the orthogonal direction(s). This amounts to minimising 
the smallest eigenvalue of the covariance matrix which 
may be preferable for testing dark energy dynamics in 
the short term. 

The redshifts, z^, and integration times, t^, for each bin 
of some optimal and near optimal surveys are shown in 
Fig. along with corresponding error bars on the Hub- 
ble rate, H{z). Typically the locally optimal geometries 
in our 5000 chains had only two (51% of all chains) or 
three redshift bins (39% of all chains) with more than 5% 
of the total survey time. Optimal geometries with either 
one or five bins were extremely rare, forming less than 
1% of all the locally optimal geometries (although single 
bin geometries deliver the thinnest error ellipses). The 
preferance for only a few bins arises because the dark en- 
ergy models we consider vary rather slowly with redshift. 



hence it is statistically preferable to constrain w rather 
than dw/dz. This conclusion may change somewhat if 
one allows very rapid evolution in w{z) which actually 
provides a very good fit to current SNIa data (Bassett et 
al.^ 2004). We found that the redshifts of the two bins 
with the most integration time were typically located at 
z < 2 (as shown in Fig. Q). 

5. CONCLUSIONS 

We have considered optimisation both of 2-d surveys 
such as CMB and weak lensing experiments and 3-d red- 
shift surveys. In a simplified optimisation of a baryon 
oscillation survey we have shown how IPSO allows sig- 
nificant gains in the statistical power of a survey can be 
achieved through optimisation, in this case a reduction 
by a factor of 6 in the error ellipse area over the unop- 
timised survey. We found that there are many diverse 
surveys with nearly degenerate figures of merit (FoM), 
as shown in Figs. & (O. This is good news since 
it allows survey designers to pick a near optimal survey 
structure that is most compatible with real-world intan- 
gibles that cannot easily be included explicitly in the 
optimisation. 

The Monte Carlo Markov Chain (MCMC) search was 
repeated thousands of times with randomly chosen initial 
survey configurations. Most of the resulting locally op- 
timal surveys divided > 90% of the survey time between 
only two or three redshift bins. A single bin leads to the 
thinnest possible error ellipse and may be appropriate 
for some experiments, particularly if the resulting ellipse 
is orthogonal to those coming from other observations. 
Alternatively, at almost the same FoM, one can choose a 
survey configuration that gives the best joint constraints 
on all the parameters simultaneously. At least for mea- 
surements of the Hubble constant alone, we found that 
typically the two dominant redshift bins should be lo- 
cated at low redshift, z < 2, as shown in Figures (@|) and 
0. This is good news for upcoming baryon oscillation 
surveys such as KAOS which will be able to probe the 
optical region z < 1.3 with high precision from earth. 



We thank Chris Blake, Eric Linder and Takahiro 
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